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FOREWORD 


In  the  course  of  conducting  research,  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  makes  use 
of  many  statistical  models  and  techniques  suited  to  a wide 
variety  of  information  gathering  and  hypothesis  testing.  The 
nature  of  conditions  in  the  field,  where  the  best  Army  data  can 
be  collected,  often  results  in  violations  of  assumptions  assumed 
to  be  critical  for  the  validity  of  specific  statistical 
operations  or  models.  The  information  provided  in  this  Techni- 
cal Paper  is  useful,  not  only  in  the  Individual  Training  and 
Skill  Evaluation  Technical  Area  but  to  experimenters  and 
analysts  in  other  areas  of  behavior  science  research  who  need 
to  determine  the  appropriateness  of  using  Analysis  of  Covariance 
(ANCOVA)  or  the  Johnson-Neyman  technique. 

The  entire  research  is  responsive  to  requirements  of  RDT&E 
Project  PE62722A777,  Individual  Training  Technology,  FY  1978 
Work  Program,  and  to  special  requirements  of  the  Deputy  Chief  of 
Staff  for  Personnel. 


AN  EMPIRICAL  INVESTIGATION  OF  THE  EFFECT  OF  HETEROSCEDASTICITY  AND  HETERO- 
GENEITY OF  VARIANCE  ON  THE  ANALYSIS  OF  COVARIANCE  AND  THE  JOHNSON-NEYMAN 
TECHNIQUE. 

BRIEF 


Requi rement : 

To  determine  the  effects  of  violating  the  assumptions  of  homoscedas- 
ticity  and  homogeneity  of  variance  on  significance  tests  associated  with 
ANCOVA  and  the  Johnson-Neyman  technique. 

Procedure: 

The  robustness  of  the  Johnson-Neyman  technique  and  analysis  of  covari- 
ance (ANCOVA)  to  violations  of  assumptions  of  homos cedasti city  and  homogeneity 
of  variance  was  tested  through  use  of  Monte  Carlo  computer  procedures.  The 
study  simulated  a one-way,  fixed-effects  analysis  with  two  treatment  groups, 
one  criterion,  Y,  and  one  covariate,  X.  Five  fixed  values  of  the  covariate 
were  selected  with  zero  mean  and  unit  variance , while  the  values  of  Y were 
vaiied  randomly  with  a constant  regression  coefficient  of  .75-  Four  com- 
binations of  group  sizes  (10 ,10 ;10  ,20 ;20  ,10 ;20 ,20 ) , five  combinations  of 
group  variances  (l,l;l,2;2,l;l,5»5,l) , and  five  forms  of  heteroscedasticity 
(combined  in  l8  different  pairs),  were  studied.  These  conditions  were 
combined  to  produce  186  different  simulated  experimental  conditions.  For 
each  simulated  condition,  3000  pseudo-random  samples  were  generated  and 
sampling  distributions  relevant  to  the  Johnson-Neyman  technique  and 
ANCOVA  were  compiled. 

Findings: 

Results  indicated  that  ANCOVA  is  robust  to  violations  of  assumptions 
of  homoscedasticity  and  homogeneity  of  variance,  both  singly  and  in  combi- 
nation, when  group  sizes  were  equal.  For  cases  of  different  group  sizes  and 
heterogeneous  variances  a predictable  bias  was  observed.  When  the  larger 
variance  was  combined  with  the  larger  group  size  the  bias  was  conservative. 
When  the  pairings  were  reversed  the  bias  was  non-conservative.  The 
Johnson-Neyman  technique  was  sensitive  to  violation  of  the  assumption 
homoscedasticity  for  both  equal  and  unequal  group  sizes.  The  effect  of 
heteroscedasticity  was  to  order  the  probability  that  any  fixed  value  of 
X would  be  included  in  a region  of  significance  in  a sequence  parallel 
to  the  form  of  heteroscedasticity.  That  is,  in  general,  as  the  variance 
for  a fixed  value  of  X increased,  the  probability  of  including  that 
value  of  the  covariate  in  a region  of  significance  increased.  As  observed 
with  ANCOVA,  the  Johnson-Neyman  technique  was  robust  to  heterogeneity  of 
variance  when  group  sizes  were  equal.  However,  when  group  sizes  were 
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not  equal  the  empirical  probabilities  were  biased  in  a non-conservative 
direction  when  the  larger  variance  was  combined  with  the  smaller  group 
size,  and  in  a conservative  direction  when  the  larger  variance  and 
larger  group  size  were  combined. 

Utilization  of  Findings: 

In  many  empirical  situations  such  as  practical  field  experiments 
conducted  within  the  Army  it  is  not  possible  to  meet  all  the  assumptions 
of  statistical  models.  If  one  or  more  of  the  assumptions  is  violated, 
the  user  has  the  choice  of  abandoning  the  model  or  proceeding  with  the 
analysis  at  some  risk.  The  results  of  this  study  may  be  used  by  the 
investigator  to  estimate  the  degree  of  bias  in  the  test  of  significance 
associated  with  either  ANCOVA  or  the  Johnson-Neyman  technique. 
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The  effect  on  Type  I error  rates  and  power  of  violations  of 
assumptions  of  mathematical  models  underlying  statistical  analyses  has 
been  studied  for  some  time.  Although  many  of  the  theoretical  consequences 
have  been  derived,  it  was  only  with  the  advent  of  high  speed  computers 
that  the  empirical  consequences  could  be  determined  with  facility.  As 
poirted  out  in  a recent  review  by  Glass  et  al.  (1972),  "the  assumptions 
of  most  mathematical  models  are  always  false  to  a greater  or  lesser 
degree."  The  purpose  of  this  study  was  to  compare  the  empirical  results 
of  certain  reasonable  violations  of  two  of  the  assumptions  underlying 
the  mathematical  models  associated  with  the  analysis  of  covariance 
(Fisher,  1932)  and  the  Johnson-Neyman  technique  (Johnson  and  Neyman , 

1936). 

Although  both  the  analysis  of  covariance  (ANCOVA)  and  the 
Johnson-Neyman  technique  incorporate  regression  methods  in  order  to 
increase  the  precision  of  an  experimental  design,  ANCOVA  assumes 
homogeneity  of  regression  whereas  the  Johnson-Neyman  technique  is  not 
based  on  this  assumption.  Furthermore,  there  are  two  current  areas  of 
great  interest  in  educational  psychology  research  where  heterogeneity 
of  regression  might  be  expected  (e.g. , Aptitude  Treatment  Interaction 
(Bracht,  1970)  and  Moderator  Variables  (Bartlett  et  al. , 1969;  Ghiselli , 
1963))-  Therefore,  the  Johnson-Neyman  technique  becomes  an  important 
alternative  to  ANCOVA. 

The  Johnson-Neyman  technique  defines  a region  along  the  covariate 
where  significant  treatment  differences  exist  between  two  groups. 

Unlike  ANCOVA,  the  Johnson-Neyman  technique  does  not  test  a hypothesis, 
but  yields  a confidence  interval.  The  region  not  included  in  the 
confidence  interval,  or  the  region  of  significance,  may  be  a single 
continuous  region  or  two  distinct  regions  where  one  group  is 
significantly  better  on  the  criterion  in  one  region,  and  significantly 
inferior  in  the  other  region. 

The  mathematical  model  for  both  ANCOVA  and  the  Johnson-Neyman 
technique  is: 


kj 


= 01+6, 


+ e. 


kj 


where 

= Y intercept  of  the  Y-on-X  regression  line  for  group  j. 

O 

J = regression  slope  of  the  Y-on-X  regression  line  for  group  j. 
e 

kj  = error  of  estimate  for  score  k in  group  j. 


The  e are  assumed  to  be  independently  and  normally  distributed  with 
KJ  2 2 

0 mean  and  homogeneous  variance,  a = a . That  is,  it  is  assumed 

y|XiJ 

that  errors  of  estimate  are  homogeneous  for  each  fixed  value  of  the 
concomitant  variable,  X , both  within  a treatment  group  (homoscedasticity ) , 
and  between  treatment  groups  (homogeneity  of  variance). 


In  addition  to  the  assumption  of  constant  error  variance,  both 
models  assume:  a linear  relationship  between  the  criterion  and  concomitant 
variable  and  that  values  of  the  concomitant  variable  are  fixed  and 
measured  without  error.  ANCOVA  makes  the  additional  assumption  of 

homogeneity  of  regression,  i.e.,  3 =*B  for  all  groups. 

J ^ 

In  many  empirical  situations  it  is  not  possible  to  meet  all  the 
assumptions  of  statistical  models.  If  one  or  more  of  the  assumptions  is 
violated,  the  user  has  the  choice  of  abandoning  the  model  or  proceeding 
with  the  analysis  at  some  risk.  Theoretical  discussions  of  a technique 
often  enable  the  investigator  to  determine  whether  his  test  is  biased 
in  a conservative  or  liberal  direction,  while  empirical  investigations 
of  violations  often  enable  the  investigator  to  estimate  the  degree  of 
bias. 


The  purpose  of  this  study  was  to  compare,  by  Monte  Carlo  methods, 
the  effects  of  violating  assumptions  of  homoscedasticity  and  homogeneity 
of  variance  on  significance  tests  associated  with  ANCOVA  and  the 
Johnson-Neyman  technique.  The  research  questions  asked  are: 

1.  Are  the  Johnson-Neyman  technique  and  ANCOVA  robust  to  the 
violation  of  the  assumption  of  homoscedasticity? 

2.  Are  the  Johnson-Neyman  technique  and  ANCOVA  robust  to  the 
violation  of  the  assumption  of  homogeneity  of  variance? 

3.  Are  the  Johnson-Neyman  technique  and  ANCOVA  robust  to  the 
simultaneous  violation  of  the  assumptions  of  homoscedasticity  and 
homogeneity  of  variance? 
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REVIEW  OF  RELATED  LITERATURE 


ANCOVA  is  an  extension  of  analysis  of  variance  (ANOVA)  and 
regression  analysis.  As  such,  assumptions  underlying  ANCOVA  include 
all  those  associated  with  ANOVA  and  regression  analysis  as  well  as  the 
assumption  of  homogeneity  of  regression.  Therefore,  results  from  studies 
of  the  robustness  of  ANOVA  co  violations  of  some  of  its  assumptions 
might  he  expected  to  extend  to  ANCOVA.  Particularly  pertinent  to  the 
current  study  is  research  on  violation  of  the  assumption  of  homogeneity 
of  variance.  The  effect  of  heterogeneous  variances  on  Type  I error 
rates  has  been  investigated  theoretically  (Box,  195^;  Scheffe,  1959)  and 
empirically  (Norton,  as  reported  in  Lindquist,  1953). 

Scheffe  (1959)  reports  the  work  of  Hsu  who  calculated  the  exact 
probability  of  Type  I error  rates  for  a two-tailed  t_ test  at  the  .05 
level  for  three  different  pairs  of2sagple  sizes  (15,5»5,3  and  7,7)  and 
9 ratios  of  population  variances  a^/Og  (0 , .1 , . 2,.  5 ,1 ,2 ,5  ,10  ,«>). 

Scheffe  (1959)  extended  Hsu's  data  to  large  samples  in  the  two-group 
case  and  studied  4 ratios  ^f  sample  sizes  (l,2,5,°°)  and  7 ratios  of 
population  variances,  (0 , .2 , . 5 ,1 ,2 ,5  ,")  • 

Scheffe  concluded  that  inequality  of  variances  has  little  effect 
on  Type  I error  rate  when  sample  sizes  are  equal,  but  has  serious  effects 
when  sample  sizes  are  not  equal.  In  general,  when  the  larger  variance  is 
associated  with  the  smaller  sample  size  the  true  probability  of  Type  I 
error  was  found  to  exceed  the  nominal  value,  and  when  the  larger 
variance  and  sample  size  were  paired  the  Type  I error  rate  was  found  to 
be  less  than  the  nominal  value. 

Box  (195*0  calculated  exact  probabilities  of  Type  I errors  for 
fixed-effect  ANOVA  F tests  at  a nominal  5 percent  level.  The  results 
were  obtained  for  3,  5,  and  7 groups,  variance  ratios  of  1:2:3;  1:1:3; 
1:1:1:1:3;  and  1: 1: 1: 1: 1: 1: 7 , and  for  11  combinations  of  equal  and 
unequal  group  sizes.  In  general,  it  was  observed  that  the  F test  is 
robust  to  the  violation  of  homogeneity  of  variance  for  equal  sample 
sizes,  but  not  for  unequal  sample  sizes.  When  the  largest  variance 
was  associated  with  the  smallest  sample  size,  the  actual  Type  I error 
rate  was  greater  than  the  nominal  value;  when  the  largest  variance  was 
associated  with  the  largest  sample  size,  the  value  of  a was  smaller  than 
nominal. 

The  robustness  of  the  F test  to  heterogeneous  variances  in  balanced 
designs  reported  by  Scheffe  (1959)  and  Box  (195*0  was  consistent  with 
an  empirical  investigation  by  Norton  (as  reported  in  Lindquist,  1953). 
Using  a Monte  Carlo  simulation  technique,  he  constructed  (by  hand) 
populations  of  10,000  cases  each  and  sampled  from  these  populations 
in  order  to  create  empirical  sampling  distributions.  In  the  phase 
of  his  study  in  which  he  investigated  heterogeneity  of  variance, 
populations  were  normal  with  equal  means,  but  different  variances 


2 

(o^  = 25,  100,  and  225).  Marked  heterogeneity  of  variance  (1:4:9) 
resulted  in  a small  but  predictable  bias  in  the  Type  I errors  and 
empirical  values  were  generally  greater  than  nominal  values. 

Little  research  on  the  effect  of  heterogeneity  of  variance  on 
robustness  of  ANCOVA  has  been  conducted.  A theoretical  paper  by 
Potthoff  (1965)  showed  that  the  sensitivity  of  ANCOVA  to  heterogeneous 
variances  depended  on  the  ratio  n a /n?a  where  n is  sample  size 

1 X1  X2 

and  a is  the  standard  error  of  the  covariate.  Three  empirical  studies 

have  £een  conducted  concerning  effects  of  violating  the  assumption  of 

homogeneity  of  variance  while  simultaneously  violating  the  assumption 

of  homogeneity  of  regression  (Peokham,  1968;  Hamilton,  1972;  and 

Mcglaren,  1972).  In  all  studies,  variance  of  the  criterion  measure 

(a  ) and  variance  of  the  covariate  (a  ) were  held  constant  while 

varying  population  regression  slopejj  fg),  ^here^y^producing  a 

concomitant  change  in  a 1 (where  a 1 = a - a 8 )•  In  the  model  for 

ANCOVA  the  assumption  o^  homogeneity  of  variance  applies  to  the  variance 

error  of  estimate  (a  1 ). 

y |x 

2 

Peckham  (1968)  varied  regression  slopes  (and  a 1 ),  number  of 
groups  and  sample  size,  although  sample  size  was  equal  between  groups 
for  all  of  his  comparisons.  Values  of  the  covariate  were  fixed  and 
were  chosen  to  conform  as  closely  as  possible  to  a normal  distribution. 
Peckham  found  that  there  were  very  small  discrepancies  in  the  actual 
and  empirical  significance  levels.  In  general,  he  observed  that  as 
the  degree  of  heterogeneity  of  the  regression  slopes  (and  heterogeneity 
of  variance)  increased,  the  empirical  rate  of  Type  I errors  was  less 
than  the  nominal  value. 

In  McClaren's  (1972)  study,  the  number  of  treatment  groups  were 
2,  3,  and  5,  and  the  sample  sizes  were  20,  30,  40 , 100,  or  200,  and 
regression  slopes  were  .1,. 2, .3, .4, .5, .6, .7, .8,  and  .9.  The  average 
slope  across  treatments  was  held  constant  at  . 5 for  l80  out  of  the 
183  simulated  conditions.  The  values  of  the  concomitant  variable 
were  fixed,  with  zero  mean  and  unit  variance.  For  equal  group  sizes, 
he  found  that  as  degree  of  heterogeneity  of  regression  increased  (and 
heterogeneity  of  variance  increased),  the  empirical  level  of  significance 
became  more  conservative.  For  unequal  sample  sizes,  the  results  parallel 
the  effect  of  violating  the  assumption  of  homogeneity  of  variance 
reported  by  Box  (1954)  and  Scheffe  (1959).  That  is,  when  the  smallest 
regression  coefficient  and  the  largest  variance  were  combined  with  the 
smallest  sample  size,  the  empirical  significance  levels  were  biased  in 
a non-conservative  direction  and  when  the  pairings  were  reversed  the 
test  was  conservative. 

Hamilton  (1972)  restricted  the  number  of  groups  to  two  and  varied 
sizes,  with  distributions  of  the  criterion  and  the  covariate  being 
bivariate  normal.  Hamilton's  results,  in  general,  parallel  the  results 
of  the  effect  of  violating  the  assumption  of  homogeneity  of  variance 
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reported  by  Box  (195^*)  and  Scheffe  (1959)-  Hamilton  found  ANCOVA 
robust  to  the  violation  of  homogeneity  of  regression  (and  variance) 
for  equal  sample  sizes,  but  observed  large  discrepancies  in  empirical 
and  nominal  alpha  levels  for  unequal  sample  sizes.  When  the  larger 
sample  si^e  occurred  with  the  larger  regression  slope,  and  therefore 
smaller  a i , the  empirical  alpha  levels  were  greater  than  corresponding 
nominal  alpha  levels.  When  the  smallej  group  size  was  paired  with  the 
larger  regression  slope  (and  smaller  a i ) , he  observed  that  empirical 
alpha  levels  were  less  than  corresponding  nominal  levels.  Hamilton's 
study  appears  to  present  evidence  for  the  generalizability  of  results 
from  the  study  of  the  effect  of  violating  the  assumptions  associated 
with  ANOVA,  as  suggested  by  Cochran  (1957)  and  Winer  (1971). 

In  one  condition,  Hamilton  studied  the  same  combination  of  equal 
sample  sizes,  number  of  groups,  and  regression  coefficients  as  Peckham 
and  as  McClaren,  but  failed  to  replicate  their  results;  a comparison 
is  presented  in  Table  1.  Whereas  Hamilton's  values  were  close  to  nominal 
alpha,  Peckham  and  McClaren  observed  a conservative  bias  in  empirical 
alpha  levels  where  group  sizes  were  equal  and  regression  slopes 
heterogeneous.  It  is  difficult  to  resolve  the  discrepancies  in  the 
results  of  these  studies.  Although  it  is  impossible  to  determine  simple 
effects  of  violating  the  assumption  of  homogeneity  of  regression  or 
variance  from  the  results  of  Hamilton  (1972),  McClaren  (1972),  and  Peckham 
(1968),  an  analytical  study  by  Atiquallah  (196U)  suggests  that  the  F 
test  of  ANCOVA  is  robust  to  the  violation  of  the  assumption  of 
homogeneity  of  regression  when  sample  size  is  large  and  the  means  of  the 

concomitant  variable  are  equal;  otherwise,  the  test  is  biased  in  a 
conservative  direction. 

There  is  no  study  examining  the  unique  effect  of  heteroscedast.icity 
on  the  robustness  of  ANCOVA.  For  an  overview  of  ANCOVA  and  the.  effects 
of  other  violations  on  ANOVA  and  ANCOVA,  comprehensive  reviews  bv 
Glass  et  al.  (l97l)  and  Elashoff  (1969)  are  available. 

The  Johns on- Neyman  technique  has  not  received  as  much  attention 
as  ANCOVA  and  little  is  known  concerning  the  effects  of  violating 
assumptions  underlying  this  statistical  method.  As  originally  presented, 
the  Johnson-Neyman  technique  was  designed  for  the  case  of  two  predictor 
variables,  two  treatment  groups,  and  one  criterion  (Johnson  and  Neyman, 
1936),  and  papers  on  the  technique  have  concerned  extension  to  cases  of* 

"n"  predictor  variables  and  "k"  groups  (Abelson,  1953;  Potthoff,  I96U). 


Table  1 


Comparison  of  Actual  and  Nominal  Levels  of  Significance  in  Simulations 
of  "True"  Experiments  Obtained  by  Peckham  (1968), 

Hamilton  (1972),  and  McClaren  ( 1972) 


Group  Sizes 

Regression 

Coefficients 

nl= 

10 ,n2= 

10 

V 

20,n2= 

20 

Nominal  Alpha 

Nominal  Alpha 

.10 

• 05 

.01 

.10 

.05 

.01 

Peckham 

■ 5,. 5 

7694 

.052 

.013 

.102 

. 01+9 

.013 

. it  , . 6 

.096 

.050 

.010 

.089 

.045 

.009 

.3, .7 

.091 

.045 

.011 

• 097 

.051 

.009 

.2,. 8 

.076 

.039 

.006 

.076 

.038 

.008 

• 1,  • 9 

.055 

.029 

.004 

.055 

.027 

.005 

Hamilton 


• 5,. 5 

.098 

.049 

.013 

.115 

.057 

.015 

.4, .6 

.099 

.053 

.011 

.103 

.052 

.on 

.3, .7 

.104 

.051 

.010 

.100 

.054 

.012 

.2, .8 

.105 

.058 

.013 

.103 

.051 

.009 

• lt.  9 

.109 

.057 

.015 

.103 

.056 

.015 

McClaren 

• 5,. 5 

.105 

.047 

.012 

.4, .6 

.099 

.049 

.009 

• 3,. 7 

.090 

.049 

.011 

.2, .8 

.074 

.037 

.010 

• l»-9 

.060 

.022 

.004 
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PROCEDURE 


The  effect  of  violating  the  following  assumptions  on  F tests 
associated  with  ANCOVA,  and  regions  of  significance  associated  with 
the  Johns on-Neyman  technique  were  investigated: 

1.  heteroscedasticity 

2.  heterogeneity  of  variance 

3.  heterogeneity  of  variance  and  heteroscedasticity 

The  study  simulated  a simple  one-way,  fixed-effects  analysis  with 
two  treatment  groups,  one  criterion,  and  one  covariate.  Five  fixed 
values  of  X,  the  covariate,  assumed  to  be  measured  without  error,  with 
zero  mean  and  unit  variance,  were  selected.  Four  combinations  of  group 
sizes  were  used:  10,10;20,20;10,20;  and  20,10;  with  an  equal  number 
of  cases  at  each  fixed  value  of  X.  All  assumptions  underlying  the  two 
methods  were  met  except  those  under  study.  The  value  of  the  regression 
coefficient  was  held  constant  at  8 = .75,  and  the  expected  values  of 
^ = $X  + a)  for  both  groups  were  -0.8485,  -0.4243,  0.0000^  +0.4243, 

+0.8485.  Nominal  significance  levels  of  .01  to  .99,  increasing  in 
steps  of  .01,  were  used  for  comparison  with  empirical  a levels. 


Heterogeneity  of  Variance 


Three  values  of  heterogeneity  of  variance  were  studied;  ay|x  was 

2 2 

set  at  3 values:  1,  2,  and  5*  As  a 
constant,  there  is  a concomitant  cha^ 


2 d 

3 | changes,  w^ile  8 and  ax  are  held 

a.Xge  in  p and  a ; resulting  values 


2 2 

are  shown  in  Table  2.  a i =1  was  paired  with  a i =1  (homogeneity  of 

y(x  ^ y|x 

2 . 2 


variance ) , 
pairs  gf  gro' 
with  a |x  = 1 and  group 


o^|  =2,  and  d^i  =5*  These  pairs  combined  wi£h  each  of  the 
oXpXsizes  produnea  11  experimental  conditions  (o  i = 1 paired 
1 and  group  size  combinations  10,20  and  20,10  are  equivalent). 


Table  2 


Values  of  a and  p where 

y 

°y  |x=0y^1_p2^  and  V1.B=.T5 


2 

°y  |x 

2 

0 

y 

p 

l 

1.56 

0.60 

2 

2.56 

0.47 

5 

5.56 

0.32 
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Heteroscedasti  city- 


Five  £orms  of  heteroscedasticity  were  studied.  The  average 
value  of  a i was  held  constant  at^l,  2,  or  5,  while  values  of  the 
error  vari&nce  for  each  fixed  X (a  i ) were  distributed  over  the 

y |XiJ 

5 fixed  values  of  X so  that  the  following  forms  were  approximated: 
2 

a.  Equal  a i for  all  X , homoscedasticity  (form  A). 
y|Xij  1J 


b.  Greatest  a 


in  the  center  with  gradually  decreasing 


y|Xij 


to  either  end  point  (form  B). 


c.  Greatest  a i at  the  largest  value  of  X and  gradually 

y|Xij 

decreasing  to  the  smallest  value  of  X (form  C). 

2 

d.  Greatest  a i at  the  smallest  value  of  X and  gradually 
increasing  to  the  largest  value  of  X (form  D). 


e.  Smallest  a i in  the  center  gradually  increasing  to  either 
y |Xij 

end  point  (form  E). 


Table  3 gives  a i at  each  fixed  point  of  X for  the  5 forms  where 
ylxi« 

2 2 J 2 

o i = 1,  a i =2,  and  a \ =5,  and  a graphic  representation  of  the 

y I x y|x  y I x 


forms  of  heteroscedasticity  is  presented  in  Figure  1.  When  combined 
in  pairs,  the  5 forms  of  heteroscedasticity  produce  25  combinations 
(including  the  null  case).  The  pairings  of  form  D with  forms  A,  B,  C, 
and  E are  the  reverse  repetitions  of  pairs  CA,  CB,  CD,  and  CE  and, 
hence,  these  were  not  investigated.  Thus,  there  were  18  combinations 
of  heteroscedasticity  which  were  combined  with  11  combinations  of 
group,  sizes  and  variance  ratios.  Where  group  sizes  and  variances  were 
equal,  mirror  images  of  combinations  were  not  investigated.  Therefore, 
the  total  number  of  simulated  experimental  conditions  was  reduced  to 
186. 
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Table  3 

2 2 

Values  for  a i where  a i =1,2,  or  5,  for  the  Five  Forms 

yix  ij  yix 

of  Heteroscedasticity  for  the  Five  Fixed  Values  of 
the  Concomitant  Variable,  X 


X. . Fixed  Value 
ij 


Form  of  Heteroscedasticity 


o | = 1 

y lx 


-1 . UlU2 

1.00 

0.1+5 

0.35 

1.75 

1.50 

-0.7071 

1.00 

0.925 

0.70 

1.20 

0.85 

0.0000 

1.00 

2.25 

1.00 

1.00 

0.30 

+0.7071 

1.00 

0.925 

1.20 

0.70 

0.85 

+1.1+142 

1.00 

0.1+5 

1.75 

0.35 

1.50 

a , =2 

y |x 


-1.1+11+2 

2.00 

0.90 

0.70 

3.50 

3.00 

-0.7071 

2.00 

1.85 

1.1+0 

2.1+0 

1.70 

0.0000 

2.00 

1+.50 

2.00 

2.00 

0.60 

+0.7071 

2.00 

1.85 

2.1+0 

1.1+0 

1.70 

+1.1+11+2 

2.00 

0.90 

3.50 

0.70 

3.00 

-1.1+11+2 

5.00 

2.25 

1.75 

8.75 

7.50 

-0.7071 

5.00 

1+.625 

3.50 

6.00 

1+.25 

0.0000 

5.00 

11.25 

5.00 

5.00 

1.50 

+0.7071 

5.00 

1+.625 

6.00 

3.50 

1+.25 

+1.1+11+2 

5.00 

2.25 

8.75 

1.75 

7.50 

The  Simulation  Procedure 


For  each  of  the  186  simulated  conditions,  the  following 
parameters  were  set: 

1.  The  fixed  set  of  values  for  X,  the  concomitant  variable. 

2.  The  number  of  observations  per  fixed  X for  each  group. 


X. 

ij 


3.  The  expectation  of  the  dependent  variable  for  each  fixed 


1*.  The  standard  deviation,  a i , of  each  fixed  X.,  for  each 

y x ij 

group.  ij 

5.  The  number  of  samples,  which  was  set  equal  to  3000. 

UNIVAC  1108  Math-Pak  (1970)  subroutines  RANDN  and  RANDU  were 
used  to  generate  random  numbers.  RANDU  computes  uniformly  distributed 
pseudo- random  real  numbers  between  0 and  1,  whereas,  RANDN  produces 
sets  of  pseudo-random  numbers  which  are  normally  distributed  with  a 
specified  mean  and  standard  deviation.  Studies  (UNIVAC  1108  Math-Pak 
Programmer's  Reference  Manual,  UP-751*2,  section  lU.3)  have  shown  that 
the  initialization  number  for  the  random  number  generator  is  critical 
for  ensuring  properties  of  randomness.  Therefore,  RANDU  was  used  to 
supply  a new  starting  number  to  RANDN  for  each  of  the  3000  samples  in 
each  run.  A check  on  randomness  of  the  generated  sequences  was  made 
for  each  of  the  l86  initialization  numbers.  Using  each  of  the 
initialization  numbers  an  empirical  F distribution  was  created  for 
the  case  where  no  assumptions  were  violated  and  group  sizes  were  each 
equal  to  10.  The  Kolmogorov-Smirnov  test  was  used  to  assess  the 
goodness-of-fit  of  these  empirical  F distributions  to  theoretical  F 
distributions;  at  a = .05,  the  test  failed  to  reject  the  hyopthesis 
of  no  difference  between  the  nominal  and  empirical  distributions  for 
all  initialization  numbers  used.  In  addition,  checks  of  the  randomness 
and  normality  of  the  samples  generated  by  RANDN  have  been  run  and 
satisfactory  results  have  been  reported  by  Hamilton  (1972). 

Goodness-of-fit  Procedure 


A total  of  186  different  goodness-of-fit  testing  situations 
were  simulated  in  this  study.  Both  ANC0VA  and  the  Johns on-Neyman 
technique  were  carried  out  for  each  experimental  condition.  Ninety- 
nine  F values  were  computed  for  each  simulation  at  significance  levels 
ranging  from  .01  to  .99  in  steps  of  .01.  The  goodness-of-fit  of  the 
empirical  F distributions  for  ANC0VA  to  the  theoretical  F distributions 
was  tested  using  the  Kolmogorov-Smirnov  one-sample  goodness-of-fit 
test  at  a = .05  (Siegel,  1956).  In  addition,  the  number  of  Type  I 
errors  was  computed  at  each  of  the  significance  levels. 
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For  the  Johnson-Neyman  technique,  the  probability  that  each  fixed 
X would  be  included  in  a region  of  significance,  as  well  as  the  total 
probability  of  obtaining  any  region  of  significance,  and  probabilities 
of  obtaining  central  versus  tail  regions  at  nominal  significance  levels 
of  .01,  .025,  .05,  and  .10  were  computed. 

RESULTS 

The  Analysis  of  Covariance 

The  results  of  the  goodness-of-fit  tests  in  each  of  the  186  Simulated 
conditions  are  presented  in  Table  1+.  The  symbol  NS  means  that  the 
Kolmogorov-Smirnov  test  failed  to  reject  the  null  hypothesis  that  there  was 
no  difference  between  the  empirical  and  nominal  F distributions;  the  symbol 
S means  that  the  Kolmogorov-Smirnov  test  rejected  the  null  hypothesis. 

The  forms  of  heteroscedasticity  are  represented  by  letters  A,B,C,D,  and 
E (see  Figure  1,  or  Table  3).  The  empirical  significance  levels  corre- 
sponding to  the  nominal  levels  of  .10,. 05,. 02,  and  .01  for  all  experimental 
combinations  are  shown  in  Table  5. 

The  Johnson-Neyman  Technique 

The  empirical  probability  of  the  inclusion  of  each  fixed  value  of 
X in  a region  of  significance  at  nominal  alpha  levels  of  . 10  , . 05  , . 025 , and 
.01  under  each  of  the  l86  simulated  experimental  conditions  is  reported  in 
Table  6.  Table  7 shows  empirical  probabilities  of  obtaining  any  region  of 
significance,  when  the  nominal  alpha  level  was  set  at  . 10  , . 05  ,. 025 , and 
.01. 
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refers  to  group. 
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Note:  A,  B,  C,  D,  E refer  to  forms  of  heteroscedasticity. 


DISCUSSION 


ANCOVA  appears  to  be  robust  to  the  violation  of  the  assumptions 
of  homoscedasticity  and  homogeneity  of  variance  both  singly  and  in 
combination,  when  group  sizes  are  equal.  In  every  condition  where 
group  sizes  and  variances  were  heterogeneous  the  goodness-of-fit 
hypothesis  was  rejected.  When  the  larger  variance  was  combined  with 
the  larger  group  size,  the  empirical  significance  levels  of  ANCOVA  were 
conservative,  and  when  the  larger  variance  was  combined  with  the  smaller 
group  size,  the  empirical  alpha  levels  were  nonconservative.  The  effect 
of  ..the  violation  of  the  assumption  of  homogeneity  of  variance  on  the 
empirical  F distribution  of  ANCOVA  parallels  the  results  obtained  by 
Norton  (as  reported  in  Lindquist,  1953)  and  Box  (195*+)  when  investigating 
the  effect  of  heterogeneity  of  variance  on  the  empirical  F distribution 
of  ANOVA.  In  addition,  the  results  obtained  by  Hamilton  (1972), 

McClaren  (1972),  and  Peckham  (1968),  where  they  investigated  the 
simultaneous  violation  of  the  assumptions  of  homogeneity  of  variance  and 
regression,  are  replicated. 

The  failure  to  find  any  condition  where  heteroscedasticity  alone 
was  responsible  for  the  rejection  of  the  goodness-of-fit  hypothesis 
leads  one  to  the  conclusion  that  the  assumption  of  homoscedasticity  is 
not  important  to  ANCOVA.  Thus,  if  variances  are  homogeneous,  the  trans- 
formation of  heteroscedastic  data  before  using  ANCOVA,  as  was  suggested 
by  Elashoff  (1969),  may  be  unnecessary. 

The  Johnson-Neyman  technique  is  robust  to  the  violation  of  the 
assumption  of  homogeneity  of  variance  when  group  sizes  are  equal.  However, 
when  group  sizes  are  unequal  heterogeneity  of  variance  biases  the 
probabilities  of  obtaining  regions  of  significance  in  the  same  direction 
as  found  with  ANCOVA.  That  is,  the  larger  variance  combined  with  the 
smaller  group  size  produces  a non-conservative  bias  and  the  larger  variance 
combined  with  the  larger  group  size  produces  a conservative  bias. 


The  probability  that  a fixed  X would  be  included  in  a region  of 
significance  was  consistently  determined  by  the  form  of  heteroscedasticity. 
If  the  shapes  of  the  probability  distributions  shown  in  Table  6 are 
compared  to  the  shapes  of  heteroscedasticity  in  Figure  1,  the  following 
conclusions  emerge: 


1.  When  a 


y|XiJ 


is  constant  across  , the  probability  that  X^ 


is  included  in  a region  of  significance  is  constant  and  equal  to  the  nominal 
alpha  level. 


2.  When  a 


xlxi 


is  greatest  for  the  central  value  of  X.  and  smallest 

* J 


for  the  tails,  the  probability  that  X is  included  in  a region  of 
significance  is  greatest  for  the  central  X and  smallest  for  the  tails, 
and  in  general,  the  probabilities  are  conservative. 
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3.  When  the  o , is  greatest  at  either  end  value  and 

y ' Xij 

progressively  decreases  to  the  opposite  end  value,  the  probability  that 

X.  will  be  included  in  a region  of  significance  follows  the  size  of 
r j 

2 

Cy.|x-  The  average  significance  level  is  close  to  the  nominal  level  of 
significance. 

2 

4.  When  0 | is  smallest  for  the  center  X. . and  largest  at 

y l*±1 

either  end,  the  probability  that  X.  will  be  included  in  a region  of 

r J 

significance  is  greatest  at  the  end  values  of  X and  smallest  for  the 

central  X...  In  general,  the  empirical  probabilities  are  nonconservative. 

J 

The  effect  of  the  form  of  heteroscedasticity  when  combined  with 
a different  form  of  heteroscedasticity  is  partially  determined  by  the 
average  variance  of  the  group.  The  form  of  heteroscedasticity  combined 
with  the  larger  variance  has  a greater  influence  on  the  form  of  the 
probability  distribution  for  fixed  values  of  X. 

Although  the  magnitude  of  the  empirical  values  of  the  probabilities 
are  further  influenced  by  the  condition  of  unequal  group  sizes  and 
heterogeneity  of  variance,  the  effects  of  the  form  of  heteroscedasticity 
outlined  above  hold  constant.  That  is,  the  relative  differences  in  the 
probabilities  associated  with  each  X^  is  fixed  by  the  form  of 

heteroscedasticity,  but  the  size  of  the  probabilities  are  biased  by  the 
combination  of  unequal  group  sizes  and  heterogeneity  of  variance. 

The  probability  of  finding  any  region  of  significance  greatly 
exceeded  the  nominal  significance  level  for  all  forms  of  heteroscedasticity. 
As  explicated  by  Potthoff  (1964)  the  Johnson-Neyman  technique  may  be  used 
to  specify  the  region  of  significance,  but  does  not  set  simultaneous 
confidence  bounds.  For  example,  one  can  say  with  95  percent  confidence 
that  for  any  specific  point  P within  the  region  of  significance  that 
there  is  a true  difference  between  the  two  groups  when  nominal  alpha 
equals  .05.  One  cannot  say  with  95  percent  confidence  that  for  all 
points  within  the  region  of  significance  there  is  a true  difference  in 
performance  for  the  two  groups. 

Although  not  directly  related  to  the  research  questions  of  the 
dissertation,  it  is  of  interest  to  inquire  how  to  set  a simultaneous 
confidence  bound  equal  to  the  nominal  alpha  level.  Potthoff  suggests  a 
method  for  setting  simultaneous  confidence  bounds  which  is  basically 
identical  with  the  defining  inequality  for  the  Johnson-Neyman  region  of 
significance  with  the  exception  that  F .is  replaced  by  (r+l)F 


l,n  +n2-4;a 


. , , . , where  r is  the  number  of  concomitant  variables.  When  this 

r+l,n  +n2-4;0’ 

procedure  was  tested  for  the  simulated  experimental  condition  of  equal 
group  sizes  (10,10)  and  equal  variances  ( ay  | x=  1 * oy  | x=  1 ) "the  probability 

of  finding  a region  of  significance  matched  the  nominal  o level  as  shown 
in  Table  8. 
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Table  8 


Empirical  Results  Obtained  Using  Potthoff's  Simultaneous 
Confidence  Bounds  When  All  Assumptions  Are  Met 
and  Group  Size  Equals  10 


Nominal  Alpha 

.10 

.05  .025 

.01 

Simultaneous  Confidence 
Coeffi cient 

.099 

.01+7  .025 

.011 

Therefore,  it  is  recommended  that  Potthoff's  procedure  for  obtaining 
simultaneous  confidence  bounds  for  the  Johnson-Neyman  technique  be  utilized 
when  one  wishes  to  obtain  a region  where  one  can  state  with  1-a  confidence 
that  there  is  a treatment  effect  simultaneously  for  all  points  within 
the  region. 

A second  approach  to  maintaining  the  experiment-wise  Type  I error 
rate  at  the  nominal  significance  level  is  to  follow  the  procedure 
suggested  by  Johnson  and  Jackson  (1959)  and  Abelson  (1953).  First,  test 
for  homogeneity  of  regression;  if  this  hypothesis  is  rejected  proceed  with 
the  Johnson-Neyman  technique,  otherwise,  use  ANCOVA. 


The  marked  effect  of  heteroscedasticity  on  the  Johnson-Neyman 
technique  suggests  that  when  heteroscedasticity  is  observed  in  the  data 
that  the  use  of  the  Johnson-Neyman  technique  is  not  appropriate  if  the 
heteroscedasticity  cannot  be  eliminated  or  minimized.  If  forms  C and  D 
are  present  in  the  data  it  may  be  possible  to  apply  a variance- 
stabilizing  transformation  (Dayton,  1970). 2 
and  testing  regression  coefficients  when  a 

has  been  discussed  by  Rutemiller  and  Bowers  (1968)  and  Levenbach  (1973). 
Rowever,  the  proper  way  of  dealing  with  heteroscedasticity  was  a problem 
presented  in  the  original  Johnson  and  Neyman  article  (1936)  and  it  has 
not  been  solved  to  date. 


The  problem  of  estimating 
I is  a 

,XiJ 


function  of  X. , 

1 3 


SUMMARY 

The  robustness  of  the  Johnson-Neyman  technique  and  analysis  of 
covariance  (ANCOVA)  to  violations  of  assumptions  of  homos cedasticity 
and  homogeneity  of  variance  was  tested  through  use  of  Monte  Carlo 
computer  procedures.  The  study  simulated  a one-way,  fixed-effects  analysis 
with  two  treatment  groups,  one  criterion,  Y,  and  one  covariate,  X.  Five 
fixed  values  of  the  covariate  were  selected  with  zero  mean  and  unit 
variance,  while  the  values  of  Y were  varied  randomly  with  a constant 
regression  coefficient  of  .75*  Four  combinations  of  group  sizes  (10,10; 

10  ,20  ;20 ,10  ;20 ,20  ) , five  combinations  of  group  variances  ( 1 ,1 ;1 ,2  ;2 ,1  ;1 , 
5;5,l),  and  five  forms  of  heteroscedasticity  (combined  in  18  different 
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pairs),  were  studied.  These  conditions  were  combined  to  produce  186 
different  simulated  experimental  conditions.  For  each  simulated 
condition,  3000  pseudo-random  samples  were  generated  and  sampling 
distributions  relevant  to  the  Johnson-Neyman  technique  and  ANCOVA 
were  compiled. 

Results  indicated  that  ANCOVA  is  robust  to  violations  of 
assumptions  of  homoscedasticity  and  homogeneity  of  variance,  both 
singly  and  in  combination,  when  group  sizes  were  equal.  For  cases 
of  different  group  sizes  and  heterogeneous  variances  a predictable 
bias  was  observed.  When  the  larger  variance  was  combined  with  the 
larger  group  size  the  bias  was  conservative.  When  the  pairings  were 
reversed  the  bias  was  non-conservative.  The  Johnson-Neyman  technique 
was  sensitive  to  violation  of  the  assumption  of  homoscedasticity  for 
both  equal  and  unequal  group  sizes.  The  effect  of  heteroscedasticity 
was  to  order  the  probability  that  any  fixed  value  of  X would  be  included 
in  a region  of  significance  in  a sequence  parallel  to  the  form  of 
heteroscedasticity.  That  is,  in  general,  as  the  variance  for  a fixed 
value  of  X increased,  the  probability  of  including  that  value  of  the 
covariate  in  a region  of  significance  increased.  As  observed  with 
ANCOVA,  the  Johnson-Neyman  technique  was  robust  to  heterogeneity  of 
variance  when  group  sizes  were  equal.  However,  when  group  sizes  were 
not  equal  the  empirical  probabilities  were  biased  in  a non-conservative 
direction  when  the  larger  variance  was  combined  with  the  smaller  group 
size,  and  in  a conservative  direction  when  the  larger  variance  and 
larger  group  size  were  combined. 

. I 
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1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TE 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI— TEX— GS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  ATSI-CTS-OR 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-CTD-DT 
1 USA  Intelligence  Ctr  & Sch.  Ft  Huachuca,  ATTN:  ATSI— CTD-CS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  DAS/SRD 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TEM 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca.  ATTN:  Library 

1 CDR,  HQ  Ft  Huachuca,  ATTN:  Tech  Ref  Div 

2 CDR.  USA  Electronic  Prvg  Grd.  ATTN:  STEEP-MT-S 
1 CDR,  Project  MASSTER,  ATTN:  Tech  Info  Center 

1 Hq  MASSTER.  USATRADOC.  LNO 
1 Research  Institute,  HQ  MASSTER,  Ft  Hood 
1 USA  Recruiting  Cmd,  Ft  Sherdian,  ATTN:  USARCPM-P 
1 Senior  Army  Adv.,  USAFAGOD/TAC,  Elgin  AF  Aux  Fid  No.  9 
1 HQ  USARPAC.  DCSPER,  APO  SF  96558,  ATTN:  GPPE-SE 
1 Stimson  Lib,  Academy  of  Health  Sciences,  Ft  Sam  Houston 
1 Marine  Corps  Inst.,  ATTN:  Dean-MCI 
1 HQUSMC,  Commandant.  ATTN:  Code  MTMT  51 

1 HQUSMC.  Commandant.  ATTN:  Code  MPI-20 

2 USCG  Academy,  New  London,  ATTN:  Admission 
2 USCG  Academy,  New  London,  ATTN:  Library 

1 USCG  Training  Ctr,  NY.  ATTN:  CO 
1 USCG  Training  Ctr,  NY.  ATTN:  Educ  Svc  Ofc 
1 USCG,  Psychol  Res  Br.  DC,  ATTN:  GP  1/62 
1 HQ  Mid-Range  Br,  MC  Det.  Quantico,  ATTN:  P&S  Div 
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1  Def  8r  Civil  Inst  of  Envrro  Medicine,  Canada 
t AIR  CRESS,  Kensington,  ATTN:  Info  Sys  Br 
1 Militaerpsykologisk  Tjenette,  Copehagen 
1 Military  Attache,  French  Embassy,  ATTN:  Doc  Sec 
1 Medecin  Chef,  C.E.R.P.A. -Arsenal.  Toulon/Naval  France 
1 Prin  Scientific  Off,  Appl  Hum  Engr  Rsch  Div,  Ministry 
of  Defense,  New  Delhi 

1  Pers  Rsch  Ofc  Library,  AKA,  Israel  Defense  Forces 
1 Minister  is  van  Defensie,  DOOP/KL  Afd  Sociaal 
Psychologische  Zaken,  The  Hague,  Netherlands 


1 USA  Aviation  Sch,  CO.  Ft  Rucker,  ATTN:  ATST-D-A 
t HQ,  USAMC.  Alexandria,  ATTN:  AMXCO-TL 
1 HQ,  USAMC,  Alexandria.  ATTN:  CDR 
1 US  Military  Academy,  West  Point,  ATTN:  Serials  Unit 
f US  Military  Academy,  West  Point.  ATTN:  Ofc  of  Milt  Ldrshp 
1 US  Military  Academy,  West  Point,  ATTN:  MAOR 
1 USA  Standardization  Gp.  UK.  FPO  NY.  ATTN:  MASE-GC 
f Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  452 
3 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  458 
1 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  450 
1 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  44t 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Acous  Sch  Div 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L51 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L5 
1 Chief  of  NavPers,  ATTN:  Pers-OR 
t NAVAIRSYA.  Norfolk,  ATTN:  Safety  Ctr 
1 Nav  Oceanographic,  DC.  ATTN:  Code  6251,  Charts  & Tech 
1 Center  of  Naval  Anal,  ATTN:  Doc  Ctr 
1 Nav AirSysCom.  ATTN:  AIR-531  X 
1 Nav  BuMed.  ATTN:  713 
1 NavHelicopterSubSqua  2,  FPO  SF  96601 
1 AFHRL  (FT)  William  AFB 
1 AFHRL  (TT)  Lowry  AFB 

1 AFHRL  (AS)  WPAF8.  OH 

2 AFHRL  (DOJZ)  Brooks  AFB 

1 AFHRL  (DOJN)  Lackland  AFB 
1 HQUSAF  (INYSD) 

1 HQUSAF  (DPXXA) 

1 AFVTG  (RD)  Randolph  AFB 

3 AMRL  (HE)  WPAFB.OH 

2 AF  Inst  of  Tech.  WPAFB,  OH.  ATTN:  ENE/SL 
f ATC  (XPTD)  Randolph  AFB 

1 USAF  AeroMed  Lib.  Brooks  AFB  (SUL-4),  ATTN:  DOC  SEC 
1 AFOSR  (NL).  Arlington 

1 AF  Log  Cmd,  McClellan  AFB,  ATTN:  ALC/DPCRB 

1 Air  Force  Academy,  CO,  ATTN:  Dept  of  Bel  Sen 
5 NavPers  & Dev  Ctr,  San  Diego 

2 Navy  Med  Neuropsychiatric  Rsch  Unit,  San  Diego 
1 Nav  Electronic  Lab,  San  Diego.  ATTN:  Res  Lab 

1 Nav  TrngCen,  San  Diego,  ATTN:  Code  9000—  Lib 
1 NavPostGraSch,  Monterey,  ATTN:  Code  55Aa 
1 NavPostGraSch,  Monterey,  ATTN:  Code  2124 
1 NavTrngEquipCtr,  Orlando,  ATTN:  Tech  Lib 
1 US  Dept  of  Labor,  DC,  ATTN:  Manpower  Admin 
1 US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1 Nat  Bur  of  Standards,  DC,  ATTN:  Computer  Info  Section 
1 Nat  Clearing  House  for  MH— Info,  Rockville 
1 Denver  Federal  Ctr.  Lakewood.  ATTN:  BLM 
12  Defense  Documentation  Center 

4 Dir  Psych,  Army  Hq,  Russell  Ofcs,  Canberra 

1 Scientific  Advsr,  Mil  Bd,  Army  Hq,  Russell  Ofcs,  Canberra 
1 Mil  and  Air  Attache,  Austrian  Embassy 

1 Centre  de  Recherche  Des  Facteurs,  Humaine  de  la  Defense 
Nationale,  Brussels 

2 Canadian  Joint  Staff  Washington 

1 C/Air  Staff,  Royal  Canadian  AF,  ATTN:  Pers  Std  Anal  Br 

3 Chief,  Canadian  Def  Rsch  Staff,  ATTN:  C/CRDS(W) 

4 British  Def  Staff,  British  Embassy,  Washington 


1 US  Marine  Corps  Liaision  Ofc.  AMC,  Alexandria.  ATTN:  AMCGS-F 
1 USATRADOC.  Ft  Monroe.  ATTN  ATRO-ED 
6 USATRADOC.  Ft  Monroe.  ATTN:  ATPR  AD 
1 USATRAOOC.  Ft  Monroe.  ATTN:  ATTS-EA 

1 USA  Forces  Cmd,  Ft  McPherson,  ATTN:  Library 

2 USA  Aviation  Test  Bd.  Ft  Rucker,  ATTN:  STEBG-PO 

1 USA  Agcy  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Library 
1 USA  Agcy  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Educ  Advisor 
1 USA  Aviation  Sch,  Ft  Rucker,  ATTN:  PO  Drawer  0 

1 HQUSA  Aviation  Sys  Cmd.  St  Louis,  ATTN:  AMSAV-ZDR 

2 USA  Aviation  Sys  Test  Act..  Edwards  AFB,  ATTN:  SAVTE-T 
1 USA  Air  Def  Sch.  Ft  Bliss.  ATTN:  ATSA  TEM 

1 USA  Air  Mobility  Rsch  & Dev  Lab.  Moffett  Fid.  ATTN:  SAVOL-AS 
1 USA  Aviation  Sch.  Res  Tng  Mgt,  Ft  Rucker,  ATTN:  ATST-T-RTM 
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